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FOREWORD 


The  U.S.  Army  strives  to  have  competent  officers  trained  in  the  art  of  battle  command. 
One  discipline  of  that  art  is  the  assessment  of  battlefield  situations.  An  experimental  program  of 
research  was  conducted  to  determine  ways  to  improve  the  cognitive  skills  used  in  situation 
assessment.  The  training  concepts  presented  in  this  report  were  a  key  component  of  the  U.S. 

Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  Fort  Leavenworth 
Research  Unit’s  research  program.  This  research  was  initiated  in  1994  at  the  special  request  of  the 
Commander  of  the  U.S.  Army  Training  and  Doctrine  Command. 

This  report  describes  the  rationale  and  intent  of  instruction  to  safeguard  against  uncertain 
or  unreliable  information  and  how  to  handle  information  conflicts.  A  companion  report,  ARI 
Technical  Report  1050,  provides  experimental  evidence  collected  on  Army  leaders  that  shows  the 
merits  of  the  instruction. 
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METHODS  FOR  TRAINING  COGNITIVE  SKILLS  IN  BATTLEFIELD  SITUATION 
ASSESSMENT 

EXECUTIVE  SUMMARY 


Research  Requirements; 

Situation  assessment  provides  the  basis  for  decisions  by  battlefield  commanders  and  their 
staff  during  both  planning  and  operations.  There  is  a  need  for  training  that  enhances  the 
component  skills  in  situation  assessment.  Such  training  should  be  based  on  a  model  of  the 
knowledge  representations  and  cognitive  processes  that  are  utilized  in  situation  assessment. 

Procedure: 

A  framework  for  battlefield  commanders’  situation  assessment  has  been  developed,  based 
on  interviews  with  active-duty  command  staff  and  on  published  work  in  cognitive  psychology. 
Principles  for  improving  situation  assessment  skills  were  developed  based  on  the  framework.  The 
situation  assessment  framework  and  principles  for  improving  situation  assessment  skills  were 
developed  based  on  the  framework  and  the  improvement  principles;  these  training  techniques  are 
described  in  the  present  report.  The  training  techniques  have  been  experimentally  tested  with 
active-duty  officers,  and  the  results  of  that  testing  are  described  in  a  companion  report. 

Findings; 

Two  training  methods  have  been  developed.  Both  methods  focus  on  metacognitive  skills 
involved  in  critiquing  and  correcting  assessments  and  plans.  The  skills  concern  finding  hidden 
assumptions  in  existing  assessments  and  plans,  and  handling  data  that  conflict  with  assessments. 

The  first  method  is  tinted  to  help  officers  find  and  assess  the  reliability  of  hidden 
assumptions  before  those  assumptions  cause  problems.  In  essence,  the  method  helps  counteract 
overconfidence.  It  includes  a  devil’s  advocate  technique  that  forces  officers  to  imagine  that  their 
exploration  of  how  potential  problems  with  an  assessment  can  be  handled,  e.g.,  by  collecting 
additional  data  to  verify  assumptions,  planning  against  the  possibility  that  assumptions  will  fail,  or 
accepting  an  assumption  as  a  known  risk. 

The  second  method  is  designed  to  help  officers  find  and  resolve  conflicting  evidence.  This 
second  method  deals  with  assumptions  after  they  cause  problems,  through  inadequate  assessments 
or  plans.  This  method  improves  the  chances  of  noticing  conflicting  data  by  sensitizing  officers  to  it 
and  providing  tools  for  handling  it  when  it  occurs.  These  tools  include  techniques  for  trying  to 
explain  the  conflicting  data  in  terms  of  the  current  assessment,  evaluating  the  plausibility  of  the 
explanations,  and  generating  alternative  assessments.  The  training  discusses  methods  for 
evaluating  competing  assessments  in  terms  of  the  plausibility  of  the  assumptions  they  require. 
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Utilization  of  Findings: 


The  training  methods  appear  to  have  wide  potential  applicability.  The  kinds  of  skills 
they  address  seem  appropriate  in  very  diverse  domains  where  initial  judgments  and  responses 
can  be  verified  and  improved  if  time,  stakes,  and  novelty  warrant.  They  may  apply  in  fields 
ranging  from  combat  to  fire  fighting  to  medical  decision  making.  Certainly,  they  appear  to  apply 
across  a  wide  spectrum  of  Army  battlefield  environments  and  force  configurations. 

Training  in  effective  thinking  skills,  as  illustrated  in  this  report,  can  be  introduced  quite 
early  in  an  officer’s  training  and  can  be  continued  and  expanded  as  training  progresses.  At  the 
basic  level,  the  training  could  be  applied  to  simpler  tactical  situations  and  decisions,  while  joint 
tactics  as  well  as  operational  and  strategic  decisions  would  be  addressed  in  more  advanced 
training  and  exercises.  Across  all  these  stages  of  training,  and  across  a  variety  of  Army 
battlefield  specializations,  the  thinking  processes  that  are  taught  would  share  a  consistent  set  of 
concepts  and  procedures.  That  same  framework  could  serve  as  an  aid  in  the  design  and 
evaluation  of  exercises  and  wargames.  The  groundwork  for  shared  situation  understanding  and 
effective  coordination  would  thus  be  laid  as  early  as  possible  and  would  evolve  into  an  explicitly 
common  framework  as  training  progressed. 
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METHODS  FOR  TRAINING  COGNITIVE  SKILLS  IN  BATTLEFIELD 

SITUATION  ASSESSMENT 

INTRODUCTION 

This  report  describes  a  training  method  for  enhancing  two  metacognitive  skills  that  are 
critical  to  officers  performing  battlefield  situation  assessment.  The  skills  concern  finding  hidden 
assumptions  in  existing  assessments  and  plans,  and  handling  data  that  conflict  with  expectations. 

The  training  methods  convey  techniques  that  have  several  attributes  to  recommend  them. 
First,  the  techniques  are  simple.  They  can  be  executed  entirely  in  the  head,  if  necessary,  and 
officers  report  that  the  techniques  are  comprehensible  and  memorable.  Second,  these  are  rapid 
techniques.  They  can  be  performed  in  minutes,  not  hours.  Third,  they  can  be  executed  by  an 
individual  or  a  group  (an  aspect  of  the  methods  that  we  will  discuss  in  a  future  report).  Third  they 
can  be  used  to  refine  plans  as  well  as  assessments.  Founh,  they  produce  a  narrative  account  of 
reasoning  that  may  benefit  individual  or  group  decision  making.  Whereas  recognitional  decision 
making  processes  are  automatic  (and  thus  unavailable  for  reflection  or  reporting)  and  analytical 
processes  are  too  abstract  to  communicate  easily  in  groups,  the  decision  making  processes  tru..ned 
here  are  both  explicit  and  concrete.  Officers  who  use  the  reasoning  processes  trained  here  should 
be  able  to  explain  their  decisions  and  participate  in  critiques  of  their  decision  making. 

The  training  techniques  are  ba.sed  on  a  cognitive  framework  tor  battlefield  situation 
assessment,  which  was  itself  based  on  the  results  of  extensive  interviews  with  active-duty 
officers  as  well  as  findings  and  research  in  cognitive  psychology.  This  report  begins  with  a 
review  of  the  battlefield  situation  assessment  framework  upon  which  the  training  is  based 
(Cohen,  Adelman,  Tolcott,  Bresnick,  &  Marvin,  1993).  The  second  section,  concerning  traini.-.g. 
opens  with  a  description  of  training  principles  that  derive  from  the  situation  assessment 
framework,  and  continues  to  describe  the  training  methods  themselves.  For  each  training  mer.nod, 
we  present  a  description  of  the  technique  meant  to  convey  the  target  skill,  an  illustration  ot  the 
technique,  and  a  summary  have  been  pilot  tested  and  evaluated  in  experimental  sessions  with 
officers  from  three  different  operational  divisions.  A  subsequent  report  will  describe  the  results 
of  these  tests. 

THE  BATTLEFIELD  SITUATION  ASSESSMExNT  FRAMEWORK 

Overview 

The  framework  that  we  have  used  to  analyze  battlefield  situation  assessment  framewor:< 
consists,  at  the  most  general  level,  of  four  components  (see  figure  1): 

1.  The  real-world  environment. 

2.  Memory  and  knowledge  structures; 

3.  Actions,  goals,  and  values;  and 

4.  Processes  for  regulating  and  monitoring  cognition 

The  figure  represents  these  components  as  arcs  that  comprise  a  circle.  The  diagram 
partitions  these  components  with  concentric  circles,  which  represent  different  aspects  of  memory, 
from  the  immediate  focus  of  working  memory  in  the  innermost  ring,  to  the  contents  of  long-term 
semantic  and  episodic  memory  in  the  outer  ring  (as  discussed  in  the  following  section). 
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Figure  1 ;  Framework  for  the  adaptive  decision-making  model. 

Thejasic  form  of  the  framework  is  inspired  by  Neisser  (1976).  In  his  concept  of  the 
perceptual  cycle,  knowledge  structures  called  schemas  actively  direct  actions  such  as  attending  to 
and  exploring  the  environment.  The  real-world  information  generated  by  that  exploration  then 
causes  changes  in  the  schemas.  These  interactions  cycle  continuously  as  the  observer  gains 
understanding  of  the  actual  world.  Connolly  and  Wagner  (1988)  extended  Neisser's  concept  to 
include  decision  cycles,  in  which  exploration  of  the  environment  causes  decision  makers  to 
refine  their  goals.  We  have  incorporated  this  extension,  and  have  added  the  iterative  role  of 
metacognition,  i.e.,  monitoring  and  regulating  one’s  own  cognitive  processes,  in  learning  both 
about  the  world  and  about  one's  own  goals. 

Neisser's  perceptual  cycle  consisted  only  of  a  path  from  knowledge  to  action  to  real-world 
and  back  to  knowledge.  Our  notion  of  cognitive  cycle,  however,  includes  many  other  pos¬ 
sibilities.  The  metacognitive  component  may  critique  and  correct  situation  knowledge  and  plan 
knowledge;  plan  knowledge  directs  sampling  of  the  environment;  these  perceptual  samples 
modify  situation  knowledge,  which  can  direct  the  composition  of  plan  knowledge.  As  just  one 
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example  of  the  potential  cycles  of  assessment,  an  initial  knowledge  structure  may  be  checked  by 
metacognitive  processes,  modified,  and  checked  again,  before  leading  to  an  action  plan,  which  is 
also  checked  by  metacognition  before  being  implemented  in  the  environment,  resulting  in  new 
knowledge.  When  time  is  limited,  metacognitive  critiquing  and  correcting  may  be  omitted. 

To  understand  the  potential  of  the  model,  it  is  important  to  examine  each  of  its  three 
cognitive  components  (excluding  the  environment)  more  closely. 

Situation  Knowledge:  Enemy  Intent  Schemas  &  Action  Strategies 

We  partition  knowledge  structure  in  this  model,  as  do  Adams,  Tenney,  and  Pew  (1991), 
into  four  segments: 

1 .  The  explicit  focus  of  working  memory  (representing  the  currently  attended  part  of  the 
situation), 

2.  The  implicit  focus  of  memory  (containing  the  full  situation  model), 

3.  Current  episodic  memory  (containing  the  history  of  the  current  problem),  and 

4.  Long-term  memory  (with  both  semantic  and  episodic  contents). 

Long-term  knowledge  takes  the  form  of  memory  schemas.  These  are  used  to  organize 
situation  assessment  information.  More  proficient  situation  assessors  appear  to  use  a  variety  of 
schemas.  Among  the  most  important  are  the  enemy  intent  schema  and  associated  action 
strategies. 

Commanders  use  enemy  intent  structures  to  organize  information  about  enemy  interests, 
strengths,  and  location,  and  describe  how  they  lead  to  intentions,  actions,  and  consequences. 

Take,  for  example,  a  situation  in  which  a  U.S.  officer  must  infer  the  intent  and  actions  of  an 
enemy  who  may  attack  along  northern  or  southern  fronts.  The  officer  knows  that  tanks  are  a 
prime  source  of  enemy  strength,  and  that  U.S.  forces  have  situated  tank  killing  systems  in  a 
southern  region.  Enemy  interests  (a  term  we  broadly  define  to  include  values,  doctrine,  and 
goals)  include  doctrine  to  avoid  opposing  strength,  such  as  northern  U.S.  tank  killing  systems. 
Finally,  enemy  location  allows  attack  in  the  north  or  south  because  terrain  is  hospitable  in  both 
areas.  These  factors  enable  the  U.S.  officer  to  infer  that  the  enemy  intent  is  to  attack  to  the  north, 
that  this  will  involve  such  actions  as  moving  artiller\'.  massing  troops,  and  command  facilities  to 
the  nonh.  The  enemy’s  intended  consequence  will  be  a  northern  breakthrough. 

.\ction  strategies  build  on  the  enemy  intent  structure.  They  include  three  modes  of 
thinking  about  enemy  intent:  proactive,  predictive,  reactive. 

In  the  proactive  mode,  the  officer’s  conception  of  the  situation  is  predicated  on  molding 
enemy  intent  by  shaping  the  battlefield,  and  specifically  by  altering  the  enemy’s  perception  of  his 
own  interests,  strength  or  location.  For  example,  an  officer’s  assessment  of  a  situation  may 
assume  that  his  planned  deceptions  will  influence  enemy  estimations  of  relative  strength,  and  that 
the  deceptions  will  persuade  the  enemy  to  adopt  an  intent  to  attack  a  force  that  is,  in  fact, 
superior  to  his  own. 

The  predictive  orientation  is  one  in  which  the  commander  uses  his  knowledge  of  enemy 
interests,  strength  and  location  to  predict  enemy  intent.  For  exaniple,  the  officer  who  understands 
the  enemy  interest,  strength  and  location  described  above  might  predict  the  northern  attack. 

In  the  reactive  mode,  a  commander  infers  enemy  intent  from  the  actions  the  enemy  carries 
out,  or  by  observing  their  consequences.  The  commander  hit  from  the  north  by  enemy  tanks 
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would,  clearly,  infer  the  enemy’s  intent,  and  possibly  reconstruct  the  causal  chain  to  that  intent 
from  enemy  interests,  strength  and  location. 

These  strategies  are  not  mutually  exclusive.  A  predictive  strategy  may  employ  reactive 
methods  (i.e.,  observations  of  enemy  actions)  to  confirm  the  predictions.  A  proactive  strategy 
may  use  predictive  methods  to  decide  what  actions  would  produce  the  desired  enemy  intent,  and 
may  use  reactive  methods  to  confirm  that  the  attempt  to  influence  enemy  intent  was  successful. 

There  are  a  range  of  other  schemas  that  officers  bring  to  bear  on  situation  assessment.  We 
will  not  discuss  them  here,  but  the  following  list  conveys  their  character; 

•  Enemy  goal  structures  describe  the  hierarchical  and  compensatory  relationships 
among  values,  goals,  and  actions. 

•  Temporal  plan  execution  structures  provide  a  more  detailed  description  of  the 
temporal  durations,  precedence  relations,  and  causal  contingencies  among  actions  and 
events. 

•  Enemy  planning  structures  describe  the  enemy  roles  and  activities  involved  in  produc¬ 
ing,  communicating,  and  implementing  plans. 

•  Terrain  structures  relate  terrain  features  to  expected  enemy  actions  and  prescribed 
friendly  actions. 


Value/action  structures 

Actions,  goals,  and  values  reflect  a  qualitatively  different  way  of  viewing  knowledge. 
They  represent  how  possible  states  of  affairs  are  valued,  whereas  situation  knowledge  represents 
how  strongly  they  are  believed.  Values  or  preferences  are  importantly  different  from  strengths  of 
belief.  They  influence  beliefs,  but  are  separate  from  them.  We  draw  on  Beach’s  (1990)  concepts 
of  the  value  image,  trajectory  image,  and  strategic  image  to  define  these  segments. 

High-level  values  are  relatively  permanent  knowledge  about  what  the  decision  maker 
regards  as  desirable,  important,  and  worth  pursuing. 

Current  goals  reflect  the  desired  sequence  of  states  from  the  present  into  the  future:  they 
are  concrete  realizations  of  high-level  values  in  the  current  situation.  Goals  thus  involve  episodic 
memory.  Goals  are  used  to  generate  plans  and  provide  the  larger  meaning  for  specific  actions. 

The  current  plan  is  the  detailed  set  of  actions  and  action  contingencies  that  the  decision 
maker  has  adopted  in  the  current  situation.  It  includes  the  specific  actions  (e.g.,  "move  up  follow¬ 
up  forces,"  "emplace  artillery,"  look  for  a  kill  zone,  etc.)  undertaken  to  realize  goals. 

The  part  of  the  plan  active  in  working  memory  is  the  immediate  focus  of  evaluation.  Such 
evaluation  may  occur  prior  to  implementation  as  part  of  the  decision  making  process,  or  during 
implementation  by  monitoring  an  on-going  action  for  its  success  in  achieving  goals. 

Processing  of  actions,  goals  and  values  can  be  either  top  down  or  bottom  up  (Beach, 
1990).  Actions  and  plans  may  be  generated  and  evaluated  based  on  goals,  and  modified  or 
rejected  if  they  fail  to  achieve  them.  Similarly,  goals  may  be  generated  and  evaluated  based  on 
values.  On  the  other  hand,  from  a  bottom-up  perspective,  goals  may  be  revised  if  no  actions  can 
be  found  to  achieve  them.  Even  high-level  values  (such  as  maintaining  the  initiative,  or  attacking 
the  enemy’s  center  of  gravity)  might  be  revised  (perhaps  rationalized  away)  if  they  are  not 
achievable  by  realistic  goals  or  actions. 
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Metacognitive  knowledge:  Overview 

Metacognition  consists  of  functions  that  monitor  and  regulate  thought.  It  has  been  defined 
as  “individuals'  knowledge  of  the  states  and  processes  of  their  own  mind  and/or  their  ability  to 
control  or  modify  these  states  and  processes”  (Gavelek  and  Raphael,  1985). 

Metacognition  is  the  focus  of  considerable  attention  by  developmental  psychologists 
(e.g.,  Flavell,  1979;  Forrest-Pressley,  MacKinnon,  and  Waller,  1985),  interested  in  how  children 
learn  to  manage  the  cognitive  activities  involved  in  reading,  comprehending,  memorizing,  and 
paying  attention.  Evidence  for  similar  processes  is  found  in  the  literature  on  expert-novice 
differences  in  problem  solving. 

While  experts  may  be  said  to  “recognize”  familiar  problems,  recognition  is  sometimes 
achieved  through  the  evaluation  of  intermediate  results.  For  example,  according  to  Larkin, 
McDermott,  Simon,  &  Simon  (1980),  physics  experts  often  construct  and  examine  a  sketch  of 
the  superficial  objects  and  relations  in  a  physics  problem  in  order  to  determine  the  next  step:  If 
the  depicted  system  is  familiar,  the  expert  may  proceed  directly  to  the  equations  required  for 
solution.  If  the  system  is  unfamiliar,  the  expert  constructs  an  idealized  representation  (i.e.,  a  free- 
body  diagram),  which  is  then  used  in  the  generation  of  solution  equations.  According  to  Chi, 
Glaser,  and  Rees  (1982),  this  qualitative  analysis  of  a  problem  is  not  a  discrete  phase  that  is 
concluded  prior  to  the  generation  of  quantitative  equations.  They  found  that  experts  returned  to. 
and  refined,  the  initial  gross  representation  when  necessary  throughout  the  course  of  the  problem. 
In  short,  experts  manipulate  the  situation  until  they  understand  it.  That  is,  they  change  their 
representation  of  the  problem  until  it  makes  contact  with  their  knowledge.  Metacognitive  skill  is 
required  in  judgments  of  familiarity  and  of  how  best  to  transform  the  problem  to  make  it  familiar. 

Metacognition  also  plays  a  role  after  the  problem  has  been  recognized  and  (apparently) 
“solved.”  Physics  experts  utilize  the  abstract  physical  representation  of  a  problem  to  verify  the 
correctness  of  their  method  and  result,  e.g.,  by  checking  whether  all  forces  are  balanced,  whether 
all  entities  in  the  diagram  are  related  to  givens  in  the  problem,  etc.  Similarly,  in  chess,  Simon 
(197-')  observed  that  some  masters  search  the  space  of  future  moves  and  countermoves  to  verif' 
that  the  moves  they  recognized  as  best  are  in  fact  in  the  subset  of  good  moves.  More  recent 
research  has  found  that  differences  in  search  skill  (i.e.,  depth,  breadth,  and  speed)  are  correlated 
with  chess  expertise  (Chamess,  1981;  Holding  and  Reynolds,  1982).  Key  aspects  of  searching  to 
verify  recognized  answers  are  metacognitive:  the  processes  of  initiating  search,  monitoring  and 
evaluating  its  results,  and  deciding  when  it  should  be  terminated. 

In  sum,  metacognitive  processes  are  crucial  in  two  phases  of  intuitive  decision  making: 

•  Constructing  a  situation  model  or  plan  when  recognition  is  uncertain. 

•  Verifying  the  results  of  recognition 

We  have  devised  a  model  that  incorporates  both  recognition  and  facilitative  processes, 
and  we  call  it  the  Recognition/Metacognition  model  (Cohen,  1993a;  Cohen,  1993b).  It  has  three 
key  components  (see  figure  2):  Quick  Test,  Critiquing,  and  Correcting.  Each  represents  a 
different  category  of  skill  in  situation  understanding  and  decision  making.  We  will  address  each 
in  turn. 
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CRITIQUING 

(a)  Discover  conflicts  (Mentally  simulate 
expectations,  compare  with  data  or^oals; 
get  others'  views;  adopt  different  points  of 
view). 

(b)  Uncover  unreliable  assumptions 
(Imagine  how  each  step  of  reasoning  could 
be  wrong). 

(c)  Test  for  incompleteness  in  model  or 
plan  (Checklist,  template,  SOP;  mental 
simulation). 


Yes 

_ Jl _ 

CORRECTING 

(1)  Collect  more  data, 

(2)  Activate  additional  parts  of 
LTM, 

--  and/or  -- 

(3)  Adjust  assumptions,  select  an 
explanation 


Revised  situation 
model  or  plan 


Figure  2:  .Metacognitive  knowledge  and  process  tlow 

Metacognitive  knowledge:  The  Quick  Test 

The  Quick  Test  is  a  gate-keeping  function  that  determines  whether  ( I)  to  engage 
critiquing  and  correcting  processes  that  might  improve  problem  recognition  or  (2)  whether  the 
current  level  of  understanding  can  (or  must)  suffice. 

This  process  answers  the  question;  Is  there  some  reason  to  think  more  about  my  current 
model  or  plan,  or  should  I  act  immediately?  The  answer  is  based  on  three  more  specific 
questions:  (I)  Do  I  have  time  before  it  is  necessary  to  commit  to  a  decision?  (2)  .Are  the  stakes  of 
an  error  high?  and  (3)  Is  there  significant  reason  to  doubt  my  initial  situation  as.sessment  or  plan? 
Quick  Test  skills  thus  involve  sensitivity  to  the  availability  of  time  and  potential  costs  of  delay; 
sensitivity  to  the  costs  of  errors  that  might  occur  if  one  acts  now  on  the  best  solution  to  date;  and 
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sensitivity  to  the  typicality  of  a  situation  and,  conversely,  to  the  presence  of  unusual  or  troubling 
features.  If  all  three  of  the  Quick  Test  conditions  are  satisfied.  Quick  Test  inhibits  the 
recognition-based  response  and  triggers  a  process  of  knowledge-based  reasoning.  If  at  least  one 
of  the  Quick  Test  conditions  is  not  satisfied,  the  initial  model  or  plan  is  accepted,  and  no 
critiquing  or  correcting  takes  place. 

The  R  /  M  model  integrates  concepts  from  other  models.  According  to  Klein  (1993), 
rapid  recognition-primed  decision  making  is  expected  under  conditions  of  high  time  pressure. 
According  to  Connolly  and  Wagner  (1988),  rapid  processing  may  occur  when  there  is  low  cost  of 
an  error.  According  to  both  Klein  (1993)  and  Rasmussen  (1993),  it  is  expected  in  highly  familiar 
situations,  that  is  from  decision  makers  with  situation-specific  expertise.  If  any  one  of  these 
conditions  is  true,  then  the  answer  to  the  corresponding  Quick  Test  question  is  “no,”  and 
correcting  and  critiquing  do  not  take  place. 

The  Quick  Test  can  be  a  relatively  explicit  and  conscious  process,  or  a  form  of 
recognitional  processing  at  a  higher  level  that  can  be  extremely  rapid  and  virtually  automatic. 

Metacognitive  knowledge:  Critiquing 

An  initial  situation  model  or  plan  may  fail  the  Quick  Test  because  specific  problems  are 
apparent  or  simply  because  the  situation  is  complex  or  unfamiliar.  In  the  latter  case,  the  next  goal 
in  metacognitive  processing  is  to  answer  the  question:  Are  there  specific  potential  problems  with 
the  current  model  or  plan?  Critiquing  is  the  stage  of  ferreting  out  such  problems. 

Critiquing  can  result  in  the  discovery  of  three  kinds  of  problems  in  the  current  model  or 
plan:  incompleteness,  unreliability,  or  conflict.  Situation  understanding  or  planning  is  incomplete 
if  conclusions  and  options  are  not  specific  and  detailed  enough.  Understanding  and  planning  may 
be  complete  but  unreliable  if  the  link  between  data  and  evidence,  or  the  link  between  actions  and 
goals,  is  uncertain  or  conditional.  Finally,  even  if  understanding  and  planning  are  complete  and 
free  of  obvious  unreliable  assumptions,  there  may  be  an  alternative,  conflicting  conclusion  that 
better  accounts  for  some  of  the  data,  or  an  alternative  incompatible  action  that  better  achieves 
some  of  the  goals. 

Some  critiquing  methods  are  general-purpose.  They  are  capable  of  uncovering  problems 
of  all  three  kinds.  Mental  simulation  and  retrieval  of  similar  cases  from  experience  are  general- 
purpose  in  this  sense.  For  example,  by  mentally  simulating  a  course  of  action,  an  officer  might  be 
able  to  see  if  the  current  plan  has  any  gaps,  if  it  reliably  achieves  goals,  and  whether  it  conflicts 
with  other  goals.  Similarly,  comparison  of  a  plan  with  an  analogous  previous  experience  might 
reveal  gaps  in  the  present  plan,  suggest  places  where  the  plan  might  not  work  reliably,  or  suggest 
alternative  actions  that  have  been  adopted  in  the  past. 

Other  critiquing  methods  are  more  specialized  in  the  kinds  of  problems  they  can  uncover. 
For  example,  decision  makers  might  use  a  checklist  or  standard  operating  procedure  to  ensure 
that  all  required  components  of  a  model  or  plan  have  been  specified  (completeness).  They  might 
adopt  a  devil’s  advocate  technique  in  order  to  ferret  out  unreliable  assumptions  in  an  assessment 
or  prediction.  Data  collection  can  determine  if  observations  are  consistent  or  conflicting  with  the 
current  situation  model  or  plan. 

Metacognitive  knowledge:  Correcting 

If  no  specific  problem  with  the  model  or  plan  is  identified  by  either  the  quick  test  or 
critiquing,  then  metacognitive  processing  in  the  current  cycle  is  complete.  But  if  a  specific 
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problem  is  found,  the  third  major  function  of  metacognition  is  enlisted:  facilitating  the 
construction  of  an  improved  model  or  plan.  Whatever  problem  is  discovered,  three  methods  are 
available  to  solve  it: 

1.  Collecting  more  data  to  fill  gaps  in  the  model  or  plan,  confirm  or  disconfirm  an 
assumption,  or  to  resolve  conflict 

2.  Activating  existing  knowledge  in  long-term  memory,  for  the  same  purposes 

3.  Adding  assumptions  to  fill  gaps  or  resolve  conflict,  and  dropping  or  replacing 
assumptions  when  they  appear  unreliable  or  conflict  with  other  information  or 
assumptions 

Metacognitive  processes  play  a  role  in  choosing  among  these  processes,  and  in  regulatin 
the  process  that  is  chosen:  (1)  in  selecting  the  amount  and  type  of  data  collection,  (2)  in  directin 
the  search  for  knowledge  in  long-term  memory,  and  (3)  in  adjudicating  among  competing 
possible  assumptions. 

Data  collection.  Sometimes  there  is  time  and  opportunity  to  collect  additional  data  to 
flesh  out  or  resolve  ambiguity  in  a  model  or  plan,  or  confirm  or  disconfirm  doubtful  assumptions. 
The  decision  to  collect  more  data  rather  than  simply  think  about  the  problem  involves 
metacognitive  judgments  regarding  the  amount  of  available  time,  the  cost  and  potential  risks  of 
data  collection,  and  the  trustworthiness  of  information  sources. 

Knowledge  activation.  Metacognitive  processes  are  crucial  in  guiding  the  serial 
activation  of  knowledge  in  long-term  memory.  Such  searches  are  conceived  of  as  controlled 
spreading  activation.  Executive  processes  determine  which  components  of  the  current  model  will 
be  attended,  thus  influencing  the  portions  of  long-term  memory  likely  to  be  activated  next 
(McClelland  &  Rumelhart,  1986).  The  values  of  the  attended  nodes  are  fixed,  or  clamped,  at  a 
high  level  of  activation  (in  effect,  accepting  them  provisionally  or  by  assumption)  in  order  to 
explore  their  implications.  In  the  next  cycle,  new  nodes  may  be  clamped,  and  so  on,  until 
knowledge  is  activated  that  satisfies  the  goals  of  the  search  (or  quick  verification  determines  that 
time  has  run  out).  Different  officers  will  attend  to  aspects  of  the  situation  in  different  orders. 

Some  may  focus  attention  on  knowledge  of  terrain,  others  on  knowledge  of  enemy  strength, 
others  on  knowledge  about  enemy  goals,  and  others  on  knowledge  of  enemy  actions. 

Metacognitive  control  may  influence  search  in  another  way,  by  adjusting  the  degree  of 
similarity  required  for  a  match  between  patterns  in  active  memory  and  stored  structures  (cf.. 
Hinton  &  Sejnowski,  1986).  When  the  threshold  is  set  low,  the  activation  net  is  cast  wide,  and 
far-fetched  ideas  have  a  significant  chance  of  being  considered.  When  the  threshold  is  set  high, 
an  idea  must  have  a  very  high  degree  of  association  with  currently  active  beliefs  to  have  a  chance 
of  being  activated.  Low  thresholds  may  be  used,  for  example,  when  schema  is  readily  available 
that  fits  all  the  data,  or  no  plan  is  readily  available  that  can  achieve  important  goals. 

Adjusting  assumptions.  If  data  collection  is  infeasible  because  of  limitations  in 
resources,  time,  or  sources  of  information,  and  if  definitive  knowledge  is  not  available  or  cannot 
be  accessed  from  long-term  memory,  the  situation  assessor  may  adopt  assumptions  to  remedy 
incompleteness  in  his  model  of  the  situation.  Metacognitive  processes  are  crucial  in  the 
interpretative  process  of  evaluating  and  revising  assumptions.  Decision  makers  think  and  act  as  if 
assumptions  were  true  until  there  is  some  reason  to  doubt  them.  Conflict  between  data  and  a 
situation  model,  or  between  two  competing  models,  provides  such  a  reason  for  doubt.  Conflict 
indicates  that  at  least  one  of  the  beliefs  involved  in  building  the  models  or  interpreting  the  data 
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was  false.  Conflict  may  thus  trigger  a  metacognitive  process  of  exposing  hidden  assumptions  and 
questioning  their  reliability.  The  process  of  revising  beliefs  to  explain  conflict  requires  a  variety 
of  metacognitive  skills:  awareness  that  conflict  exists,  an  ability  to  uncover  implicit  assumptions 
that  have  created  the  conflict,  sufficient  awareness  of  the  structure  of  one's  beliefs  to  identify  the 
assumptions  that  are  central  to  a  variety  of  models  and  plans,  and  sensitivity  to  factors  that 
indicate  unreliability  of  assumptions,  for  example,  by  recalling  past  episodes  in  which  the  same 
beliefs  led  to  a  conflict.  Finally,  the  process  of  assumption  revision  calls  for  balancing  the 
plausibility  and  the  power  of  the  resulting  models  and  plans. 

TRAINING  METACOGNITIVE  SKILLS 
Training  Principles 

We  derive  from  the  R/M  model  several  principles  for  training  (to  be  described  at  greater 
length  in  another  report).  These  principles  contrast  with  those  that  reflect  different  models  of 
decision  making.  For  example,  if  we  view  decision  making  as  an  analytical  process,  a  perfectly 
appropriate.stance  in  some  circumstances,  then  it  follows  that  instruction  should  focus  on  the 
transformations  required  to  represent  problems  within  the  analytical  model  (for  example, 
quantitative  representations  of  probabilities  and  utilities).  Practice  may  be  partitioned  such  that 
trainees  study  parts  of  the  method  before  attempting  to  execute  the  whole.  Examples  should 
come  from  varied  domains  to  demonstrate  the  generality  of  the  method,  and  they  should  be 
presented  in  a  graduated  sequence  progressing  from  problems  that  clearly  afford  the  analytical 
technique  (that  is.  they  require  little  in  the  way  of  transformation)  to  those  that  superficially  seem 
inappropriate  to  the  technique  (that  is,  they  are  quite  difficult  to  completely  and  accurately 
represent). 

If  decision  making  is  taken  to  be  a  recognitional  process,  again  a  reasonable  assumption 
in  some  situations,  then  the  following  inferences  about  training  hold.  Instruction  should  focus  on 
goals,  environmental  conditions,  and  actions.  In  procedural  terms,  instruction  should  convey  the 
following  type  of  knowledge:  if  your  goal  is  X  amidst  events  Y,  then  perform  action  Z.  Practice 
conditions  should  be  realistic  to  promote  accurate  recognition  and  repetitive  to  facilitate 
automatic  performance.  Examples  should  be  representative  of  the  domain,  though  feedback  may 
be  made  more  explicit  and  immediate  to  speed  training. 

Training  that  assumes  an  adaptive  model  of  decision  making,  such  as  the  R/M  model, 
should  first  of  all  convey  the  iterative  nature  of  problem  solving  —  from  recognition  to 
metacognitive  checks,  critiques,  and  corrections,  back  to  recognition.  It  must  also  aim  to  sensitize 
trainees  to  domain-specific  cues  concerning  the  time  constraints,  stakes,  and  familiarity  of 
problems,  as  well  as  the  nature  of  conflict,  completeness  and  reliability  in  specific  domains.  It 
must  convey  methods  of  critiquing  situation  assessments  and  plans,  and  of  making  corrections 
that  will  enhance  understanding.  Practice  should  be  designed  to  make  metacognitive  processes 
explicit.  Such  practice  might  involve  reciprocal  teaching  (Paiincsar  and  Brown,  1984),  team 
exercises,  explicit  labeling  of  one’s  own  activities,  listening  to  experts  label  their  thought 
processes,  or  critiquing  the  performance  of  peers.  All  of  these  methods  help  trainees  make  public 
otherwise  hidden  cognitive  activities.  Making  principles  explicit  may  helps  trainees  transfer  what 
they  have  learned  to  varied  settings  (Collins,  Brown,  &  Newman,  1989).  In  addition,  it  may  be 
appropriate  to  give  trainees  control  over  practice  parameters  that  cue  metacognitive  activities; 
these  include  time,  immediacy  of  feedback,  quality  of  feedback,  and  difficulty.  Exercises  should 
employ  non-routine  cases,  as  these  discourage  simple  recognitional  solution  and  encourage  the 
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use  of  metacognitive  skills.  We  have  attempted  to  bear  these  guidelines  in  mind  while 
developing  training  to  teach  metacognitive  skills. 

The  training  methods  we  have  developed  focus  on  two  metacognitive  skills.  Both 
methods  are  designed  to  hone  officers’  skills  in  critiquing  and  correcting  assessments  and  plans. 
The  first  method  is  intended  to  help  officers  find  and  assess  the  reliability  of  hidden  assumptions 
before  those  assumptions  cause  problems.  In  essence,  the  method  helps  counteract 
overconfidence.  The  second  method  is  designed  to  help  officers  find  and  resolve  conflicting 
evidence.  This  second  method  deals  with  assumptions  after  they  cause  problems,  through 
inadequate  assessments  or  plans.  The  second  method  may  be  a  cure  for  underconfidence,  or 
confusion.  We  describe  each  of  the  training  methods,  below. 

Finding  Hidden  .4ssumptions 


The  Method 

The  method  of  finding  hidden  assumptions  consists  of  four  steps. 

1 .  Select  a  critical  part  of  your  assessment  —  even  if  you  are  confident  of  it. 

2.  Imagine  that  an  infallible  crystal  ball  (or  some  other  “perfect”  intelligence  source) 
tells  you  that  this  part  of  your  assessment  is  wrong. 

3.  Explain  how  this  part  of  your  assessment  could  be  wrong. 

4.  Imagine  that  the  crystal  ball  now  tells  you  that  your  explanation  is  wrong  and  directs 
you  back  to  step  3.  (Continue  until  your  explanations  cover  a  variety  of  different 
mechanisms,  which  seem  representative.) 

The  product  of  this  exercise  is  a  list  of  ways  an  assessment  could  be  wrong.  To  maintain 
confidence  in  the  assessment,  one  must  be  confident  that  none  of  these  is  the  case.  It  may  be 
possible  to  dismiss  some  possibilities  as  implausible.  Others  it  may  be  advisable  to  verify  by 
collecting  additional  data.  Still  others  can  be  planned  against.  The  end  result  may  be  that  one  is 
as  confident  in  the  assessment  as  when  you  started.  But  such  confidence  is  now  earned  by  taking 
seriously  the  possibility  that  the  assessment  is  wrong,  asking  how  that  could  happen,  and 
handling  each  possibility.  Unless  each  possible  exception  to  the  original  assessment  can  be  dealt 
with  in  some  way,  the  original  confidence  may  not  be  justified.  This  simple  method  reveals 
hidden  assumptions  that  underlie  assessments  upon  which  plans  are  based. 

An  Illustration 

The  following  illustration  —  drawn  from  an  interactive  classroom  exercise  —  illustrates 
the  method. 

An  officer’s  assessment  includes  the  claim  that  the  enemy  will  cross  the  river  at  location 
X.  He  bases  this  claim  on  arguments  concerning  the  distance  the  enemy  must  travel  to  his 
supposed  objective  via  point  X,  the  shallow  depth  of  the  river  at  X,  and  better  concealment 
opportunities  along  the  bank  at  X.  The  officer  is  confident  of  this  assessment;  however  stakes  are 
high  and  there  is  time  to  critique  the  assessment,  so  he  does  so.  Following  the  method,  above,  he 
imagines  that  a  crystal  ball  tells  him  that  the  enemy  will  not  cross  at  location  X,  and  demands 
that  he  explain  this  failure  in  his  interpretation  of  the  evidence.  He  cycles  through  steps  three  and 
four  to  generate  the  following  list  of  explanations  (or  negations  of  assumptions): 
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•  “The  enemy  anticipates  that  our  force  will  be  at  location  X.” 

•  “The  enemy  will  detect  the  movement  of  our  force  to  location  X.” 

•  “There  are  good  crossing  sites  that  we  missed.” 

•  “The  enemy  doesn’t  know  how  good  a  location  X  is.” 

•  “The  enemy  doesn’t  have  any  river  crossing  assets.  He  can’t  cross  the  river  at  all.” 

•  “The  enemy’s  river  crossing  assets  are  so  good  that  he  can  cross  elsewhere.” 

•  “The  enemy  has  a  large  enough  force  that  he  can  accept  casualties  crossing 
elsewhere.” 

•  “The  enemy’s  objectives  are  different.  He  doesn’t  need  to  cross  at  all.” 

•  “The  enemy  will  use  air  assault  to  get  across  the  river,  rather  than  cross  it.” 

These  nine  ideas  are  exceptions  to  the  officer’s  recognitional  assessment  of  the  situation. 
Some  may  be  implausible  (such  as  the  suggestion  that  the  enemy  can  afford  high  casualties  in  a 
river  crossing).  Others  may  require  defensive  planning.  For  example,  the  enemy  could  anticipate 
the  current  location  of  own  forces,  but  moving  forces  surreptitiously  would  protect  against  this. 

Table  1:  Example  of  exceptions  to  the  interpretation  of  evidence,  and  methods  of  handling  the 
exceptions. 


Assessment:  The  enemy  will  cross  at  X. 

Exceptions 

Responses 

“The  enemy  anticipates  that  our  force  will  be  at 
location  X.” 

Place  our  forces  elsewhere  then  move  at  the  last 
minute. 

‘The  enemy  will  detect  the  movement  of  our  force  to 
location  X.” 

Same  as  above. 

There  are  good  crossing  sites  that  we  missed.” 

Consult  with  a  specialist  in  river  crossings  or  scout 
enemy  movements  towards  other  crossing  sites. 

‘The  enemy  doesn’t  know  how  good  a  location  X 
is.” 

Dismiss  this  as  implausible  given  recent  intel  on 
enemy  surveying  and  scouting  activity, 

The  enemy  doesn’t  have  any  river  crossing  assets. 
He  can’t  cross  the  river  at  all.” 

Dismiss  this.  It  presents  no  problem  if  true. 

The  enemy’s  river  crossing  assets  are  so  good  that 
he  can  cross  elsewhere.” 

Collect  intel  to  verify  or  disprove. 

The  enemy  has  a  large  enough  force  that  he  can 
accept  casualties  crossing  elsewhere.” 

Dismiss  this  as  implausible  given  current,  reliable 
intel. 

The  enemy’s  objectives  are  different.  He  doesn’t 
need  to  cross  at  all.” 

Adopt  a  contingency  plan. 

The  enemy  will  use  air  assault  to  get  across  the 
river,  rather  than  cross  it.” 

Accept  this  risk. 
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Table  1  includes  several  responses  to  the  explanations  generated  above.  These  responses 
are  of  several  types:  testing  the  truth  of  an  assumption  through  recall  of  evidence  or  data 
collection,  adopting  proactive  strategies,  adopting  contingency  plans,  and  accepting  the 
assumption  as  a  known  risk. 

We  have  found  this  technique  of  critical  thinking  to  be  fast  and  highly  productive.  It  is 
particularly  effective  for  officers  when  they  are  confident  of  their  assessments.  It  raises  issues 
that  force  officers  to  earn  their  own  confidence  by  strengthening  their  assessments  and  plans  to 
compensate  for  potential  weaknesses. 

Training  Materials 

The  text  used  in  training  this  method  is  presented  in  Appendix  A  on  pages  A-5  to  A- 17. 
The  text  begins  with  a  motivating  statement  concerning  the  method:  “Hidden  assumptions  are  a 
source  of  weakness  in  situation  assessments,  and  a  source  of  risk  in  the  courses  of  action  based 
on  them.”  We  then  present  the  method  in  the  form  of  a  flow  chart,  and  apply  it  to  a  military 
scenario.  Next,  some  ways  to  use  the  output  of  the  method  are  briefly  described.  We  then  present 
a  final  motivating  statement,  and  a  practice  exercise  with  sample  solutions. 

This  unit  is  designed  to  be  used  in  conjunction  with  brief  lectures  and  interactive 
exercises  based  on  pretest  materials  or  the  officers’  personal  experiences.  A  particularly  effective 
classroom  exercise,  when  trainees  have  had  some  operational  experience,  is  to  ask  for  a 
description  of  a  situation  in  which  someone  was  very  sure  of  something  (e.g.,  that  the  enemy 
would  cross  the  river  in  a  certain  location).  When  the  question  is  put  this  way,  the  example 
usually  involves  a  situation  in  which  the  assessment  turned  out  correct.  The  instructor  then 
“brings  out”  the  crystal  ball,  which  says  that  this  assessment  is  wrong,  and  the  class  is  asked  to 
generate  ways  in  which  that  could  happen.  The  exception  conditions  are  listed  and  the  class  is 
asked  to  evaluate  their  seriousness  and  plausibility,  and  asked  what  they  would  do  to  handle  each 
possibility. 

The  next  unit  of  training  deals  with  situations  in  which  assessments  turn  out  to  be  wrong. 

Handling  the  Unexpected 


The  Method 

Despite  careful  assessments  and  planning,  officers  sometimes  find  themselves 
confronting  unexpected  events.  The  second  training  method  we  have  developed  is  designed  to 
help  officerThandle  events  that  conflict  with  a  current  assessment,  rather  than  simply  disregard 
or  discount  them.  At  the  same  time,  an  unexpected  event  does  not  necessarily  imply  that  the 
current  assessment  is  wrong.  In  this  training,  officers  begin  by  explaining  events  in  terms  of  the 
current  assessment,  but  if  these  explanations  are  implausible,  they  alter  the  assessments 
themselves.  The  procedure  consists  of  these  steps: 

1.  Notice  unexpected  events. 

2.  Explain  how  an  unexpected  event  could  occur  even  if  your  current  assessment  is 
correct.  (If  there  have  been  previous  unexpected  events,  try  to  find  the  simplest 
plausible  explanation  of  all  of  them.) 

3.  Evaluate  the  plausibility  of  your  explanations. 
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4.  If  the  explanations  are  implausible  (and  if  time  is  available  and  the  stakes  are  high), 
take  up  another  assessment  and  repeat  the  process  from  step  1 . 

The  first  step  is  crucial.  Noticing  unexpected  events  is  more  likely  if  officers  have 
thought  about  the  events  implied  by  their  assessments.  (Event  templates  and  war-gaming 
facilitate  this.)  This  training  may  also  increase  the  likelihood  of  noticing  unexpected  events,  in 
two  ways:  by  sensitizing  officers  to  conflicting  evidence  through  presentation  of  examples,  and 
by  teaching  them  methods  for  handling  it  when  it  occurs. 

In  some  cases,  the  first  step  of  the  method  can  be  the  last.  That  is.  it  is  sometimes  enough 
simply  to  notice  an  unexpected  event.  This  is  the  case  if  there  is  no  time  to  analyze  the  event,  if 
the  stakes  of  the  situation  are  below  some  threshold,  or  if  one  easily  recognizes  the  implications 
of  the  event.  However,  unexpected  events  should  be  remembered:  Officers  may  want  to  revisit 
them  if  additional  unexpected  events  occur  whose  implications  are  not  as  clear,  or  if  the  stakes  of 
the  situation  should  change. 

To  execute  the  second  step,  explaining  the  event,  the  officer  can  use  two  tools.  The 
cr\'stal  ball  (or  “perfect”  information  source)  is  productive.  It  instructs  the  officer  to  imagine  that 
the  assessment  is  true,  and  that  he  must  explain  the  surprising  event.  Secondly,  trainees  often  find 
it  useful  to  focus  the  technique  by  looking  for  explanations  within  each  of  the  categories  of 
METT-T  (mission,  enemy,  troops,  terrain  and  weather,  and  time  available).  For  example,  the 
trainee  may  consider  how  an  event  might  be  explained  in  terms  of  actions  of  his  own  troops,  the 
terrain  or  weather,  enemy  tactics,  enemy  equipment,  and  so  forth. 

The  product  of  these  methods  is  a  set  of  ways  in  which  the  current  assessment  could  be 
true,  despite  the  conflicting  evidence.  In  order  to  hold  onto  the  assessment,  then,  it  is  necessary  to 
accept  at  least  one  of  these  conditions  as  plausible.  If  each  possibility  is  implausible,  or  is 
disconfirmed  by  actual  data  collection,  then  alternative  assessments  should  be  considered. 

The  third  step  of  the  method,  thus,  is  an  evaluation  of  the  plausibility  of  the  explanations. 
An  important  feature  of  this  step  is  to  look  at  explanations  for  all  the  conflicting  data  that  have 
been  observed,  not  just  the  most  recent  item.  The  total  set  of  explanations  of  all  this  conflicting 
data  must  be  judged  plausible  if  the  assessment  is  to  be  kept  unchanged.  The  most  plausible 
overall  account  must  consist  of  individually  plausible  explanations,  but  it  should  also  be  simple 
—  i.e.,  not  require  a  large  number  of  independent  assumptions.  For  example,  an  account  of 
events  that  relies  heavily  on  enemy  deception  to  explain  all  of  the  unexpected  events  is 
maximally  simple,  but  not  necessarily  plausible  because  deception  may  be  a  dubious  explanation 
for  several  specific  events.  A  plausible  account  must  balance  simplicity  against  the  credibility  of 
individual  explanations. 

The  fourth  step  in  the  method  involves  taking  up  another  assessment  for  consideration. 

To  generate  a  new  assessment,  officers  can  focus  on  the  unexpected  evidence  by  itself  as  a 
catalyst  for  a  new  assessment.  Officers  examine  the  unexpected  events  they  have  encountered 
and  answer  such  questions  as,  “What  do  you  make  of  these  events?”  or  “What  do  these  events 
usually  mean?”  Thus,  we  spur  the  creative  process  by  restricting  the  evidence  from  which 
inferences  must  be  drawn.  Once  a  new  assessment  is  formulated,  however,  the  officer  must 
consider  the  rest  of  the  evidence,  i.e.,  the  evidence  that  supported  the  original  assessment.  These 
events  will  conflict  with  the  new  assessment.  Thus,  these  events  must  be  explained,  using  the 
methods  described  above,  in  a  manner  that  is  consistent  with  the  new  assessment.  These 
explanations  must  then  be  evaluated  for  plausibility  and  simplicity.  The  old  assessment  and  the 
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new  assessment  may  be  compared  by  asking  how  convincing  a  “story”  can  told  in  support  of 
each;  in  other  words,  how  plausible  and  simple  are  the  assumptions  that  each  entails. 

In  some  cases,  a  new  assessment  that  is  stronger  than  either  of  the  previous  ones  can  be 
generated  by  combining  their  strengths.  In  this  strategy,  the  decision  maker  focuses  on  the 
assumptions  that  the  assessments  require,  dropping  the  unreliable  ones  and  adopting  relatively 
plausible  elements  from  both  “stories.” 

Illustration 

An  example  is  presented  here  in  a  table  format  used  during  training  (see  Table  2).  The 
table  consists  of  an  assessment  concerning  the  enemy’s  plan  of  attack  (on  the  top  row),  and 
several  events  that  appear  to  contradict  the  assessment  in  the  left-hand  column.  Across  from  each 
conflicting  event,  in  the  right-hand  column,  are  explanations  that  have  been  generated  to  explain 
the  conflicting  events  to  make  them  consistent  with  the  assessment. 


Table  2;  Table  format  used  to  train  both  finding  hidden  assumptions  and  handling  the 
unexpected. 


Assessment:  The  enemy  plans  a  diversionary 
attack  in  the  north,  and  a  main  attack  in  the  south. 

Events 

Explanations 

Southern  enemy  force  has  destroyed  a  major  bridge 
to  his  front 

*[Troops]  Perhaps  it  was  our  own  forces,  not  the 
enemy,  that  destroyed  the  bridge. 

[Enemy  (deception)]  Perhaps  the  destruction  is  a 
deception  to  make  you  think  the  main  attack  will  not 
be  in  the  southern  sector. 

[Enemy  (mistake)]  Perhaps  the  destruction  of  the 
bridge  was  a  mistake  by  the  enemy. 

Enemy  has  initiated  radio  silence  in  the  north  and 
the  south. 

[Troops]  The  enemy  is  not  responsible  for  radio 
silence,  Our  interdiction  campaign  may  have 
destroyed  critical  enemy  radio  facilities. 

'[Enemy  (deception)]  The  enemy  is  concealing  the 
location  of  the  attack  by  instituting  radio  silence 
everywhere. 

[Enemy  (equipment)]  The  enemy’s  C2  equipment 
has  failed. 

Reinforcements  have  arrived  in  the  north. 

[Enemy  (deception)]  The  enemy  is  “showing”  arrival 
in  the  north  before  shifting  forces  stealthily  to  the 
south. 

'[Enemy  (tactics)]  The  enemy  may  be  putting  green 
forces  on  line  in  the  north  to  execute  a  secondary 
attack. 

[Enemy  (mistake)]  The  northern  movement  of 
reinforcements  may  be  a  mistake. 
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In  this  example,  the  decision  maker  has  used  the  crystal  ball  and  METT-T  categories  to 
generate  the  interpretations  of  each  event  (in  the  right  column).  The  METT-T  category'^ of  each 
explanation  is  presented  in  brackets.  Explanations  marked  with  an  asterisk  are  those  the  officer 
considers  the  most  credible. 

The  most  plausible  three  explanations  for  the  events  in  the  example  concern  troops, 
enemy  deception,  and  enemy  tactics.  However,  it  may  be  improbable  to  the  officer  that  all  three 
of  these  explanations  are  independently  true.  Thus,  for  example,  he  may  choose  a  somewhat 
simpler  account,  in  which  a  single  cause  —  our  own  aggressive  interdiction  campaign  — 
accounts  for  the  destruction  of  bridges  and  radio  silence,  and  a  second  assumption  concerning 
enemy  tactics  explains  reinforcements  in  the  north.  This  account  balances  credibility  and 
simplicity. 

Thus  far,  this  illustration  employs  the  first  three  steps  of  the  method  presented  above. 
However,  an  officer  may  find  that  he  cannot  construct  a  plausible  account  of  all  of  the 
unexpected  events.  There  may  be  too  many  events  to  explain.  Suppose  that  the  example  were 
extended  to  include  another  unexpected  event:  the  movement  of  two  enemy  motorized  rifle 
regiments  from  the  south  to  the  north.  Suppose  further  that  every  explanation  for  this  event  (such 
as  enemy  deception  or  errors)  significantly  diminishes  the  plausibility  of  the  account.  That  is, 
there  is  no  credible  and  simple  story  that  supports  the  assessment  that  the  enemy  plans  a  main 
attack  in  the  south  and  a  diversionary  attack  in  the  north.  In  this  case,  a  new  assessment  is  in 
order,  per  the  fourth  step  of  the  procedure  outlined  above. 

The  officer  examines  the  unexpected  events  to  date  and  generates  a  new  assessment,  in 
this  case  that  the  enemy  plans  its  main  attack  in  the  north.  He  then  fills  the  events  column  of  a 
table  (identical  in  structure  to  Table  2)  with  data  that  appear  to  contradict  the  new  assessment. 
For  example,  he  may  recall  that  the  southern  terrain  is  superior  for  movement  and  that  a  southern 
port  is  a  likely  enemy  objective.  To  each  unexpected  datum,  he  applies  the  method  of  handling 
unexpected  events.  That  is,  he  generates  multiple  interpretations  of  each  datum,  and  selects  from 
among  all  explanations  the  few  that  tell  a  plausible  story. 

Training  materials 

The  text  used  in  training  this  method  is  presented  in  Appendix  A  on  pages  A- 18  to  A-51. 
It  is  divided  into  three  sections.  The  first  introduces  the  method,  the  second  applies  it  to 
explaining  unexpected  events  (using  the  first  three  steps  of  the  method),  and  the  third  concerns 
revising  the  situation  assessment  (beginning  with  the  third  and  fourth  steps,  then  cycling  back  to 
the  first). 

The  introduction  begins  with  a  motivating  statement,  an  outline  of  the  four-step  method 
in  flow-chart  form,  and  a  scenario  consisting  of  a  situation  description,  an  assessment,  and  an 
update  describing  an  unexpected  development,  upon  which  illustrations  of  the  method  are  based. 

The  next  section,  on  explaining  unexpected  events,  draws  on  the  scenario  to  fill  out  a 
table  containing  slots  for  the  situation  assessment,  unexpected  events,  and  explanations  of  those 
events.  Instructions  and  tips  are  presented  in  floating  captions  beside  the  table,  as  it  is  filled  in 
over  several  pages.  This  section  ends  with  a  statement  of  the  conditions  of  plausibility  under 
which  the  assessment  should  be  revised. 
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assumptions  that  support  it,  and  to  actively  test  it  by  predicting  future  events.  The  section  ends 
with  a  motivating  summary  of  the  method,  a  new,  practice  scenario  with  sample  solutions,  and  a 
pointer  to  an  interactive  class  exercise. 


CONCLUSION 

The  present  report  is  part  of  a  series  of  reports.  Their  purposes  are  to; 

•  describe  a  framework  for  battlefield  situation  assessment  skill  in  terms  of  cognitive 
theory 

•  derive  implications  of  the  framework  for  improving  battlefield  situation  assessment 
performance, 

•  generate  techniques  for  training  battlefield  situation  assessment,  and 

•  experimentally  test  the  effectiveness  of  the  training  techniques. 

The  training  methods  described  above  only  illustrate  the  kinds  of  training  techniques  that 
might  be  developed  for  enhancing  critical  thinking  skills.  Nevertheless,  they  appear  to  have  wide 
potential  applicability.  The  kinds  of  skills  they  address  seem  appropriate  in  very  diverse  domains 
where  initial  judgments  and  responses  can  be  verified  and  improved  if  time,  stakes,  and  novelty 
warrant.  They  may  apply  in  fields  ranging  from  combat  to  fire  fighting  to  medical  decision 
making.  Certainly,  they  appear  to  apply  across  a  wide  spectrum  of  Army  battlefield  environments 
and  force  configurations. 

Training  in  effective  thinking  skills,  as  illustrated  in  this  report,  can  be  introduced  quite 
early  in  an  officer’s  training  and  can  be  continued  and  expanded  as  training  progresses.  At  the 
basic  level,  the  training  could  be  applied  to  simpler  tactical  situations  and  decisions,  while  joint 
tactics  as  well  as  operational  and  strategic  decisions  would  be  addressed  in  more  advanced 
training  and  exercises.  Across  all  these  stages  of  training,  and  across  a  variety  of  Army  battlefield 
specializations,  the  thinking  processes  that  are  taught  would  share  a  consistent  set  of  concepts 
and  procedures.  That  same  framework  could  serve  as  an  aid  in  the  design  and  evaluation  of 
exercises  and  wargames.  The  groundwork  for  shared  situation  understanding  and  effective 
coordination  would  thus  be  laid  as  early  as  possible  and  would  evolve  into  an  explicit  common 
framework  as  training  progressed. 
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APPENDIX 


The  following  pages  include  the  training  materials  described  in  the  text, 
divided  into  the  following  sections; 


Introduction 

23 

Finding  Hidden  Assumptions 

25 

Handling  the  Unexpected 

45 

The  material  is 
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One  of  the  most  difficult  tasks  for  any 
commander  or  staff  officer  is  to  accuratel 
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Situation  Assessment  Training  Introduction 


This  l^raining  focusGS  on  two  skills  crucial  to  situation  assossmont. 
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EXAMPLE 
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"Explain  how  it  could  be  that  the 
enemy  will  not  cross  at  X,  despite 
the  distance,  depth,  and 
concealment." 


30 


Situation  AsSGSSmGnt  TrsiniriQ  |  l  incling  Hidden  Assumptions 


^  .52 

fPi 

■S  ^ 
•£  ^ 

^  c 
o  «  .o 

CD  ■£ 

CoS 
c  b  £ 
A  o  -5 

01  CO  8- 


CD 

> 

A  ^ 
A>  2 

§  2 

nv  <li 


«i  rs  i 

t  ?|  I 

%  ^  ^  $ 

••S  2  I 


is  ^ 
Q.  5^ 
5"  ^ 

Ui  ^ 
«J  03 

2  ^ 

•52  c 

■I  ^ 
^  2 

I" 


cn  ^  1 


S'  ^ 
.£  ^75 

73 

'§0.9 

-  o  c 
01  <0  cn 


</)  A 

5  § 

E  ^ 
0)  ^ 
C: 

S  §■ 

0)  O 

i?  ^  ■ 
K  CO 

..  C 

K  CO  , 
=tfc  ^ 

c  ^ 

o  ^ 

§  ^ 

is  cS 

Q.  Q) 

SiS 

CD  2 

^  § 

2 


CD  I 

-c:  I 

^  I 

CD  3 

CD  i 


CO  i 

CO  n 


CO  a 
.30  o 

A  ^ 

9^  .CO 


CO  « 
(D  CO  I 

i.  •*-  I 

2  CO  i 

:§'  O  I 
0^1 
Cl  ^ 

^■2  I 

2  "Q  1 

c  CD  I 
^  C  I 

CD 

iE  o 

r  -6 

S  ^  I 

^  iS  I 

c  I 

.2-^1 
■S  ^  i 
g  2  I 

I  I 

uj  i 


:2  . 
CC3  '  . 

?  ^ 
.b=  CO 

CO  p 


P  o 

rry 


Situation  Assessment  Training  Finding  Hidden  Assumptions 


Now,  Evaluate  the  possible  Exceptions 

— : - - - 1 - - - — - - - - 

This  method  doesn't  have  to  make  you  less  sure  of  your  conclusion.  Just  because 
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Example:  Evaluating  the  Exceptions 
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EXERCISE 
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Evaluating  the  Exceptions 
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Exercise 
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Apply  the  technique 
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Situation  Assessment  T raining  Handling  the  Unexpected 


A  good  situation  assossmont  allows  you  to  oxplain  past  ovonts, 
predict  future  events,  and  devise  a  sound  course  of  action.  But 
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There  are  two  types  of  responses  you  can  make  to 
r'lexpected  events  and  information,  such  as  reports  of  past 
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Handling  the  Unexpected 
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We  will  show  you... 
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enemy  plans  a  diversionary  attack  in  the  north,  and  a  main  attack  in  the  south 
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Assessment:  The  enemy  plans  a  diversionary  attack  in  the  north,  and  a  main  attack  in  the  south 
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Assessment:  The  enemy  plans  a  diversionary  attack  in  the  north,  and  a  main  attack  in  the  south 


O 

c 

o  c 


D.  ® 

<D  :S 

^  3 

T3  yj 
CO  <D 

.52  Si 


CO 

to 

to 


$ 


<D 

O) 

.*2 

<D 

x: 


(/) 

c 

o 

CO 

c 

Q. 

X 

LU 


(0 

5  ai 

CO  ;d 

£  <u 
0)  S 

&2 

P  m 


C  C 

o 

o  ^ 

2  o 
(0  c 

<B  _ 

—  o 

CO  CO 
Q.  C 
CO  <0 

-E  C 
o  '5 
E 

<D 

.§ 

Q. 

0)  -  = 

O  £ 

(U  ^ 

TD  3 
^  O 

>,  >s  ^ 

E  03  o 
P  o 


to 

Q. 

to 

ir 

i 

c 

(D  03 

03 

iS  .c 
(0  *- 


03 

E  CO 
?  i 

LU  E 


55 


Situation  Assessment  T raining  Handling  the  Unexpected 


56 


Situation  Assessment  T raining  Handling  the  unexpected 


o 

ca 

o 

T3 

o 

r. 

w 

3 

2 

c 

■d 

o 

c 

5 

o 

>V 

o 

Q. 

o 

o 

(U 

*D 

© 

-B 

3 

O 

V) 

© 

O) 

:d 

©' 

o 

c 

© 

S 

© 

c 

© 

<0 

o 

o 

© 

is 

W 

© 

x: 

-C  CO 


-it: 

o 

o 


cc 

c 

CL 

X 

CD 


CL 

0 

0 

o 

WJ— 

CO 

c 

Z3 

0 

0 

x: 

0 

o 

E 

"D 

0 

0 

0 

> 

0 

E 

o 

Q 

E 

o 

0 

C 

o 

o 

0 

o 

JC 

-4«-' 

0 

c 

c 

C 

0 

rD 

~ 

0 

-1— • 

> 

0 

CL 

Z3 

O 

o 

0 

1- 

X 

o 

0 

0 

c 

<D 

> 

0 

0 

C 

O 


a; 


CO 

CO 


57 


Situation  Assessment  T raining  Handling  the  unexpected 


ASS6SSm0nt.  The  enemy  plans  a  diversionary  attack  In  the  north,  and  a  main  attack  in  the  south 
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ASSGSSmGnti  The  enemy  plans  a  diversionaiy  attack  in  the  north,  and  a  main  attack  in  the  south 
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enemy  plans  a  diversionary  attack  in  the  north,  and  a  main  attack  in  the  south 
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Don't  forget  your  original  reasons! 


63 


Situation  Assessment  T raining  Handling  the  unexpected 


AsS6SSm6nt!  The  enemy  plans  to  launch  his  main  attack  in  the  north. 
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Exercise 
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Assessment: 
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Guerrillas  will  avoid  trails  in  valleys  and  on  ridge  lines. 
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Assessment: 


73 


Situation  Assessment  T raining  Handling  the  unexpected 


Assessment: 
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A  new  I  I - — _ _ _ 

assessment  I  Assessment:  enemy  will  use  valley  trails  and  ridge  lines  rather 

and  some  |  i  than  hillside  contours. 
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Assessment: 
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Assessment; 
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